The specification relates to a robot-human interaction, and as a more specific, non-limiting example, technology for the object handover from robot to human.
In existing approaches, when a robot receives a command to handover an object to a person, the robot executes a predetermined action sequence that hopefully results in the person taking the object from the robot. The robot executes the one executable handover action that was preprogrammed by a designer. In some instances, the parameters of the handover action can be adjusted, but the action itself does not change. In some instances, there are interfaces by which a person can manually identify actions for the robot perform. However, in these instances, the robot is not autonomous—does not switch between actions itself and does not learn when one action should be performed in favor of another.
One approach at improving a robot's handover actions includes defining or identifying human activity by the objects a person is interacting with, and vice versa (e.g., see Hema S. Koppula and Ashutosh Saxena, “Anticipating Human Activities using Object Affordance for Reactive Robotic Response,” IEEE Transactions on Pattern Analysis and Machine Intelligence; and Yun Jiang, Hema S. Koppula, and Ashutosh Saxena, “Modeling 3D Environments through Hidden Human Context,” in Sexena PAMI 2015). Under this approach, different objects afford different types of human activity, and if those affordances can be learned, then a robot/computer system can identify the human activity from the objects. The above works use conditional random fields to anticipate human activities from a combination of skeletal data and detect objects in relation to the human position. However, reliance on skeletal data is often infeasible or unreliable due to the skeletal information often not being available or a portion of the skeletal information being obscured by other objects.
Another approach focuses on human preferences to modify handover actions (e.g., see M. Cakmak, S. S. Srinivasa, Min Kyung Lee, J. Forlizzi, and S. Kiesler, “Human Preferences for Robot-Human Hand-over Configurations,” in Intelligent Robots and Systems (IROS), 2011). This approach emphasizes trajectories, not handover locations. While a trajectory can be helpful for determining handover actions, the location at which the handover action occurs is not evaluated by this approach. Rather, the approach assumes that the handover location is known, which is typically not the case in dynamic environments, and only the person's expectations of how the robot moves need to be matched in order to improve the quality of the handover.
Another approach focuses on determining the location of the handover by combing different affordance data (e.g., see A. K. Pandey, and R. Alami, “Affordance Graph: A Framework to Encode Perspective Taking and Effort Based Affordances for day-to-day Human Robot Interaction,” in Intelligent Robots and Systems (IROS), 2013). The emphasis of this approach is on incorporating a reachable area map from the current human position and then adding the perspective of the person and the robot (e.g., the ability of the human and the robot to see that location). The result is an affordance graph, where the effort required to handover the object at each location is estimated through reachability and visibility. This approach however assumes that the person is not going to move significantly from their current location to retrieve the object, which, like the foregoing approach, is typically inapplicable in a true dynamic environment.
Another approach considers the mobility and willingness of a human participant in determining the best location at which to handover the object (e.g., see Jim Mainprice, Mamoun Gharbi, Thierry Sim'eon, and Rachid Alami, “Sharing Effort in Planning Human-Robot Handover Task,” in IEEE RO-MAN, Paris France, 2011). This approach considers that if the person is capable of assisting the handover, and doing so either will speed up the process significantly, or can be done with minimal cost to the human, then many individuals would choose to move to meet the robot. However, this approach fails to consider the environments, the human activity, or the object being handed over. Additionally, although people can change their position in order to share the effort in a human robot handover task, they may be otherwise occupied or unwilling to wait for the robot to navigate through a difficult environment.
Therefore, a need exists for a handover action from robot to human that is more autonomous, dynamic, and adaptive in different situations.